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ABSTRACT Jf feedback from educational research is not properly 

related or interpreted, it can often be more detrimental than 
beneficial. Five factors that should be kept in mind regarding 
feedback are: (1) varying interpretations of research finding 

depending on the user, (2) ethical-human aspects, (3) validity 
problems, (4) interaction effects, and (5) contamination of the 
original sample. Two case studies illustrating some of the above 
mentioned problems are presented. (CK) 



ED057078 



US DEPARTMENT OF HEALTH. 
EDUCATION & WELFARE 
OFFICE OF EDUCATION 
THIS DOCUMENT HAS BEEN REPRO- 
DUCED EXACTLY AS RECEIVED FROM 

the person or organization orig- 
inating IT POINTS OF VIEW OR OPIN- 
IONS STATED DO NOT NECESSARILY 
REPRESENT OFFICIAL OFFICE OF EDU- 
CATION POSITION OR POLICY- 



DISADVANTAGES OF FEEDBACK 



Richard C. Williams 
University of California, Los Angeles 









Paper presented at the annual meeting of the 
American Educational Research Association, 
New York, New York, February, 1971 



ERIC 1 



DISADVANTAGES OF FEEDBACK 



Richard C. Williams 
University of California, Los Angeles 

For many years school administrators and teachers have been voicing 
dissatisfaction with the work of educational researchers. Some of the most 
common criticisms are that researchers interrupt the practitioners' work in 
the schools; researchers often do not attack the really important problems 
facing education, consequently there is precious little payoff, in the form of 
improved educational practice, that results from all its time and money spent 
on educational research. 

Recently a new criticism has emerged, namely, that researchers do not 
report back their findings to those who participated in studies. Practitioners 
state that they agree to participate in a study, they willingly give of their time 
and energy to cooperate with the research effort, but when the research is 
completed, no effort is made to tell them the results of the research. In short, 
they received no "feedback. " 

In response to this criticism, some researchers are making promises, 
when they secure the right to collect data, that they will "feed back” the 
findings to those who participated in the study. 

The title of my presentation today is "The Disadvantages of Feedback. ” 
Perhaps a better title would be "Some Cautions to Those Who Promise Feedback. 
I wish to make it clear at the outset that I am quite favorably disposed to feed- 
back, but I should like to describe some problems one faces when he feeds back 
that he does not encounter when he engages in standard research reporting. 



And that one should consider these problems before he promises feedback so 
that he does not obligate himself to something he cannot or should not perform. 

Perhaps it would be useful at this time for me to distinguish between 

standard research reporting and feedback. 

Standard research reporting, as I am using the term, means the publi- 
cation of research findings at meetings such as this, in journals or monographs, 
or other special purpose publications. A conscientious effort is made to 
disguise the identity of those who participated in the research study. 

Feedback, on the other hand, means the direct reporting of research 
findings to those who participate in a study. The ability of the researcher to 
cloak the findings in anonymit is oftentimes greatly reduced. Feedback can 
occur as part of some intervention strategy (as some will describe today), or 
it can be presented at the completion of the study. 

In discussing these concerns, let me first outline some considerations a 
researcher should keep in mind when he contemplates whether or not to promise 
feedback of his research findings. I will then b rW' ’ -wo studies, one 

in which direct, open feedback would seem appropriate, and another study wnere 
feedback would seem unwise or at least it should be carefully structured. 

First, what are some considerations the researcher should keep in mind 
when considering feedback? 

1. Your findings have a greater likelihood of "making a difference. " 
As we all know, one can complete marginal or exploratory research studies in 
education, and even if they are reported to a larger research audience, the 
results will likely have little or no immediate impact. The self- professed 



limitations you place on your research will generally be acknowledged by other 
researchers and they will interpret your work in the light of their knowledge of 
the problems that plague your research efforts. Such interpretative precautions 
are not generally heeded by practitioners, and they are more likely to accept 
your work at face value. And you, the researcher, have to accept your share 
of the responsibility for whatever actions, however unjustified, may be taken on 
the basis of your feedback. 

2. A second problem is closely related to the first; it is what I will 
label the ethical-human problem. This is a concern, particularly when the 
anonymity of the respondents cannot be effectively maintained. Suppose you 
find that a particular administrator or teacher, or combination thereof, bear a 
major responsibility for some undesirable condition in a school. Do you report 
this to everyone, to the person's superordinate, or only to the person involved? 
As I indicated earlier, your findings can make a difference in the lives of indi- 
viduals. If you a. .... rr^uiwk, unfident that your findings are true, then you 
may be relieved of some burden of responsibility, but if there is reasonable 
disagreement over the validity of your findings, or if your conclusion- are par- 
tially or conditionally true, or if your findings are subject to careful interpre- 
tation, then you may be exacerbating a problem that already exists, you may be 
perpetuating a problem that no longer exists, or you may indeed be creating a 
new problem. For example, we solicited the opinions of the teachers in a 
school about the adequacy of leadership being provided by the principal. Now 
this school had just gone through the agonies of a teachers ’ strike. The princi- 
pal of the school had tended to personalize the militant actions of tie teachers 
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and had become temporarily quite autocratic in his behavior. The teachers had 
reacted negatively to the principal’s changed leadership style and the teachers 
negativism was reflected in the data. Sometime later, after the antagonisms of 
the strike had receded and the relationships between the principal and teachers 
had returned almost to their former state of mutual confidence, we found the 
teachers' view of their principal as a leader had become once again more posi- 
tive. Had we not returned to the school, and had we reported the earlier findings 
at a later date, we may have seriously, and unknowingly, affected the relation- 
ship between the principal and his teachers by reporting this "unstable" finding. 

3. There is the validity problem. How do you know you are measuring 
what you claim to be measuring? Some variables are what I would refer to as 
hard data; that is, there is considerable agreement as to what the data n* sai 
Examples are financial or enrollment data. Many other data, however, are what 
I would call "soft"; that is, they are constructs over which there is considerable 
difference of opinion as to what those constructs are and how they should be 
measured. Examples are such constructs as morale, intelligence, leadership), 
and social systems. Often these "soft" data are dynamic and very complicated. 

4. There is an interactive effect. Suppose ’ ou decide "feedback” is 
desirable or necessary, but in an effort to lessen the human- ethical problem, 
you decide to give the participants in a research study, prior to the study, a 
fuller picture of your research intent than you would normally provide. By so 
doing, if the subjects agree to the study as described, and to the feedback you 
propose, you will be relieved of your ethical responsibilities. This may lighten 
your guilt problems, but in outlining the study in some detail you may weu 



influence the validity of the responses you receive from the research participants. 
That is, the subjects may manipulate their answers to your inquiries in such a 
way as to present themselves in a favorable light, during the feedback, and by so 

doing they will invalidate your findings. 

5. Finally, once you have fed back some types of data to a school, that 
school should be considered "contaminated" insofar as it can be considered a 
"normal school". The interactions or introspection that may take place because 
of the feedback may be such that the school simply cannot be used for further 
research. This is not a problem, of course, if you are unconcerned about 
further research in that school or if inferences from subsequent research are 

not going to be made to other schools. 

Now let me describe briefly two studies, one in which feedback would 
present few of the problems I have just described, and another study that is 
heavily burdened with them. 

Suppose one were to conduct research to explore the relationship between 
the income of parents of children in school attendance areas, the measured 
academic achievement of pupils in those schools, and the financial expenditures, 
by she district per school. And let us suppose that you find that the school with 
the lowest family income has the lowest achievement, and gets the lowest 
budgetary allocation from the district. The "feeding back" of such findings, 
even showing each school staff and parents where their particular school ranks 
on this scale, is not too difficult a problem. First, the data are relatively hard, 
though the achievement tests may be subject to some controversy. Second, it is 
difficult to attach blame on specific individuals for this condition, and this 



indirectness lessens the chance of inappropriate actions based on your findings. 

On the other hand, suppose you are going to examine the relationship 
between a principal's oerceived leader behavior and teacher morale in several 
schools, and you promised feedback would include reporting back to each 
participating school how they compared with other schools. Let's suppose 
further that your hypothesis of a relationship between perceived poor leader 
behavior and low morale is supported, and that School X is a perfect example 
of that finding. How do you report to School X? Whatever caveats you may 
place on your findings may be ignored and people may act or react to your 
findings in unanticipated ways. And you cannot evade the responsibilities for 
their acts, however unjustified. 

As we know, the constructs of "leadership" and "morale are subject to 
some controversy, and there are those who will question the appropriateness of 
the instruments you have chosen to use. Also, these constructs are embedded in 
a complex, dynamic social system that is difficult to measure. For example, 
suppose you find a principal's perceived leader behavior is poor. Have you 
accounted for the historical dimension of that finding? Perhaps he is improving 
his leader-behavior and your feeding back of negative findings may well affect 
negatively his self-image, or the staff’s or the superordinate's perception of 
him may be unfairly reinforced, and his poor leadership may become a perma- 
nent self-fulfilling prophecy. Thus his future career may be seriously in 
jeopardy. Finally, if you attempt to explain the research in any detail prior to 
data collection, to lessen the ethical problem, you may so contaminate the 
study that the findings will be of questionable validity. 



In summary, feedback may indeed be an important activity for those who 
wish to maximize the impact of their research or who wish to increase the 
interest of practitioners in research. It is, however, not an unmixed blessing. 
And those who promise feedback might best consider the possible unintended 
results their feedback may have on the lives and careers of those who partici- 
pated in the study and subsequently received the feedback. 
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